In a stream processing system, a stream processing unit may communicate with other processing units to receive events for processing. This may be done through either a direct communication from one processing unit with another or through a publication-subscription model where a processing unit subscribes to certain events and receives the events when they are published.
As the scale of a stream processing system increases, e.g., where millions of processing units are involved, it may be impractical to utilize direct communication techniques between processing units. While the publication-subscription model might seem feasible, the traditional publication-subscription model requires a processing unit to explicitly submit event subscriptions before receiving any information related to the events. This may be a problem because, in some cases, it may be impossible for a processing unit to predict what events to subscribe to before certain events occur. For example, a processing unit configured to process events occurring on a map of a game application does not know which game to subscribe to until a game has started on that map. In such situations, in order for the processing unit to receive the events it is interested in, the processing unit has to subscribe to all possible events, which again becomes impractical and inefficient in a large-scale stream processing system.
It is with respect to these and other considerations that the disclosure made herein is presented.